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Preface 

The  current  pilot  study  deals  with  the  representation  of  uncertainty  in  C  I  infonnation 
(Situation  Awareness)  systems,  which  are  regarded  as  partial  representations  of  real-world 
objects  and  events.  Decision  makers  may  face  uncertainty  due  to  individual  circumstances. 

■J 

However,  much  uncertainty  results  from  the  limited  ability  of  C  I  information  system  to 
fully  represent  real-world  objects  and  events.  For  the  commander,  this  uncertainty 
constitutes  a  baseline  to  which  we  refer  as  basic  uncertainty.  In  the  pilot  study  we  review 
related  literature,  propose  new  ways  for  categorizing  uncertainty,  discuss  possible  methods 
for  attenuating  the  negative  effects  of  uncertainty  and  propose  further  research. 

This  material  is  based  upon  work  supported  by  the  European  office  of  Aerospace  Research 
and  Development  (EOARD),  London,  England  under  Contract  Number  FA8655-03-1- 
3041.  Lt  Col,  Valerie  E.  Martindale,  Chief,  Life  Science  and  Human  Effectiveness,  was  the 
contract  monitor.  Mr.  Gilbert  Kuperman  of  the  Air  Force  Research  Laboratory’s  Human 
Effectiveness  Directorate  (AFRL/HECS)  served  as  the  contracting  officer’s  technical 
representative. 

Any  opinions,  findings  and  conclusions  or  recommendations  expressed  in  this  material  are 
those  of  the  authors  and  do  not  necessarily  reflect  the  views  of  the  European  Office  of 
Aerospace  Research  and  Development  (EOARD). 


The  authors  wish  to  thank  Lt.  Col  Martindale  for  her  support  and  to  Mr.  Gilbert  Kuperman 
for  his  support,  guidance  and  invaluable  contributions  throughout  this  study. 
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Abstract 

In  recent  years  military  forces  have  been  striving  to  adapt  network  centric,  information 
superiority-enabled  warfare  concepts  (Leedom,  2001).  One  of  the  practical  implications  is 
the  development  of  technology  for  acquiring,  processing  distributing  and  displaying 
battlefield  information.  Command,  Control,  Communication  and  Intelligence  (C  I) 
information  systems  are  gradually  deployed  to  all  operational  levels,  from  the  highest 
command  echelons  to  individual  shooters.  It  is  argued  however,  that  despite  these 
developments,  the  level  of  uncertainty  in  the  battlefield  may  not  be  much  lower  than  it  was 
in  the  past. 

The  present  report  deals  with  the  representation  of  uncertainty  in  C  I  information  (Situation 
Awareness)  systems,  which  are  regarded  as  partial  representations  of  real-world  objects  and 
events.  Decision  makers  may  experience  uncertainty  because  of  insecurity,  lack  of 
experience,  etc.;  much  uncertainty,  however,  results  from  the  limited  ability  of  C  I 
information  system  to  provide  full  and  accurate  representations  of  real-world  information. 
This  uncertainty  constitutes  a  baseline  to  which  we  refer  as  basic  uncertainty. 

The  term  "information  world"  is  used  to  describe  the  real-world  information  that  a 
commander  or  a  command  team  need  during  various  stages  of  a  representative  C  I  mission. 
This  information  is  the  basis  for  decision  making  which,  most  often,  takes  place  under 
highly  stressful  conditions.  The  human  operator  is  unable  to  directly  acquire  real-world 
information  and  has  to  rely  on  information  that  is  symbolically  represented  by  C  I 
information  systems,  and  which  is  inherently  partial  and  inaccurate  (i.e.,  uncertain). 

The  cognitive  concepts  of  Situation  Awareness  (SA),  Sensemaking  and  Naturalistic 
Decision  Making  (NDM)  have  developed  during  recent  years  in  an  effort  to  gain  better 
psychological  understanding  of  human  operation  and  performance  in  real-world  situations, 
particularly,  situations  that  involve  complex,  human-machine  systems.  These  theoretical 
frameworks  are  used  to  analyze  the  CT  environment  and  CT  information  systems.  Two 
methods  for  sorting  uncertainty  in  C  T  are  presented:  sorting  by  source  and  sorting  by  level 
of  uncertainly.  First,  sorting  by  sources  of  uncertainly  which  may  be  technology  related 
(acquisition,  processing,  and  display  systems)  or  user  related  (experience,  confidence,  etc.), 
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where  system  related  sources  are  defined  as  basic  uncertainty.  Second,  sorting  by  the  level 
ofSA,  Perception,  Comprehension  or  Projection  (Endsley,  1995),  that  is  supported  by  the 
information.  Then  we  analyze  various  methods  of  coping  with  uncertainty,  focusing 
primarily  on  basic  uncertainty.  Basic  uncertainty  may  sometimes  be  reduced  by  improving 
information  quality  and  representation  techniques,  however,  the  focus  of  this  study  is  on 
how  to  deal  with  information  about  uncertainty ;  what  are  the  pros  and  cons  of  presenting  or 
not  presenting  the  user  with  information  about  known  sources  of  basic  uncertainty.  Some 
possible  approaches  and  techniques  for  coping  with  the  remaining  uncertainty  and 
attenuating  its  negative  consequences  are  presented. 

Finally,  ideas  for  further  research,  focusing  on  the  representation  of  uncertainty,  are 
outlined  and  briefly  discussed. 
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Introduction 

The  technology  for  acquiring,  processing,  distributing  and  displaying  battlefield 
information  is  constantly  developing.  The  military  forces  are  striving  to  parallel  the 
infonnation  revolution  in  the  commercial  sector  by  adapting  network  centric  warfare 
concepts  (Leedom,  2001),  defined  as  "an  information  superiority-enabled  concept  of 
operations  that  generates  increased  combat  power  by  networking  sensors,  decision-makers, 
and  shooters  to  achieve  shared  awareness,  increased  speed  of  command,  higher  tempo  of 
operations,  greater  lethality,  increased  survivability  and  a  degree  of  self-synchronization" 
(Alberts,  Garstks  &  Stein,  1999).  C  I  information  systems  are  actually  being  developed  in 
the  framework  of  several  ongoing  projects.  For  example,  the  US  Army  FBCB2  system 
(Talcott,  Martinez,  Bennet,  Stansifer  &  Shattuck,  2001);  the  Lockheed  Martin  contract  to 
develop  a  Joint  battle-space  situational  awareness  capability  designed  to  serve  as  the 
primary  common  operational  picture  for  all  four  military  services  (Lockheed  Martin,  May 
6,  2003);  MAGIS  -  Marine  Corps  Air  Ground  Intelligence  System  (Intelligence  Department 
Headquarters,  UA  Marine  Corps,  2003);  The  USAF  "Cyber  Warrior"  information  system 
(Kuperman,  Whitaker  &  Brown,  2000). 

Winning  the  information  war  may  be  the  key  to  victory  in  war  (Kuperman,  2001).  On  the 
other  hand,  lack  of  information  or  unreliable  information  can  break  the  command  and 
control  process  and  disrupt  any  offensive  initiative,  causing  one’s  forces  to  remain  one  step 
behind  the  enemy  (Flach  &  Kuperman,  2001). 

As  an  important  part  of  the  concept  of  information  superiority,  C I  information  systems  that 
previously  had  been  used  only  by  high  levels  of  command  or  on  board  major  weapon 
systems  (e.g.,  bombers),  are  gradually  being  deployed  to  the  shooters’  level  (Leedom, 
2001).  Despite  these  developments,  however,  the  level  of  uncertainty  in  the  battlefield  may 
not  be  much  lower  than  it  was  in  the  past.  The  modem  battlefield  may  contain  a  greater 
than  ever  number  and  variety  of  objects,  many  of  which  may  be  dynamic.  In  terms  of  the 
US  Army  'cognitive  hierarchy'  (Headquarters  Department  of  the  Army,  1996,  from 

■j 

Kuperman  2001,  p.  232)  C  l  systems  may  support  the  levels  of  data  and  information  but 
may  not  contribute  much  to  knowledge  and  understanding. 
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Figure  1  presents  a  schematic  illustration  of  this  notion. 
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Figure  1:  The  C  I  information  system  may  provide  ample  amounts  of  high  quality  data  but 
less  information,  knowledge  and  understanding.  The  width  of  the  line  symbolizes  quantity 
and  its  darkness  symbolizes  reliability  (e.g.,  there  may  be  much  reliable  data  but  only  little 
and  unreliable  support  for  understanding). 
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The  present  report  deals  with  the  representation  of  uncertainty  in  C  I  information  systems. 
A  C  I  decision  maker  may  experience  uncertainty  because  of  insecurity,  lack  of  experience, 
etc.  Much  uncertainty,  however,  results  from  the  inability  of  C  I  infonnation  systems  to 
provide  a  full  and  accurate  representation  of  necessary  information.  This  uncertainty 
constitutes  a  baseline  to  which  we  refer  as  basic  uncertainty. 

The  next  section  presents  some  of  the  circumstances  under  which  commanders  make 
decisions:  their  "information  world",  the  nature  of  C  I  information  systems  and  some  initial 
notions  regarding  the  representation  of  uncertainty. 
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The  "information  world"  and  its  representations 

The  "information  world"  of  the  commander 

Commanders  plan,  command  and  control  missions  that  take  place  in  the  real  world.  The 
term  "information  world"  is  used  to  describe  the  combined  sources  of  information  that  are 
necessary  for  the  perfonnance  of  these  tasks.  The  "world"  can  be  divided  into  four 
dimensions: 

■  The  physical  world  (geography,  atmosphere). 

■  Friendly  forces  (command  levels,  commanded  forces,  own  unit,  other  units,  neutral 
forces). 

■  Enemy  (sites,  targets,  threats). 

■  Mission  stages  (planning/rehearsal,  mission  perfonnance,  debriefing). 

Table  1  presents  an  example  of  the  possible  components  in  the  information  world  of  a 

-i 

commander  or  a  command  team,  in  a  C  I  aerial  strike  command  center  (i.e.,  an  air 
operations  center).  Based  on  Endsley's  (1995)  definition  of  situation  awareness,  (see  next 
section  -  "Theoretical  background")  it  may  be  useful,  for  some  information  elements,  to 
indicate  the  necessary  levels  of  commanders'  knowledge:  Perception  (detection,  recognition 
and  identification  of  major  features);  Comprehension  (significance,  intentions,  capabilities, 
etc.);  and  Projection  (Where  will  it  be?  and  What  will  it  be  doing  in  the  near  future?). 
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Table  1:  Example  of  elements  in  the  "information  world"  in  a  C  l  aerial  strike  command 
center. 
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Type  of  information 

Level  of  knowledge 

Category 

Details 

Example 

Perception 

Comprehension 

Projection 

Enemy 

Sites 

SAM  sites 

Location 

significance 

Future  status 

Deployment 

Mobile  SAM 

Targets 

SCUD 

launchers 

Location 

recognition 

Hit  probabilities 

Future  status 

Aerial  threats 

Enemy  fighter 
aircraft 

location 

Level  of  threat 

Future  status 

Ground 

threats 

SAM 

location 

Level  of  threat 

Future  status 

Mission 

stages 

Planning 

Monitor 

intelligence 

Relevant 

items 

Significance 

Effects  on 
mission 

Plan  mission 

Prepare  & 

distribute 

orders 

Command  & 
control 

Monitor 

forces 

Location 

status 

Significance 

Effects  on 
mission 

Refresh 

commands 

Compare  to 
planning 

Debriefing 

(not  in  scope) 

Stress  and  ucertaintv  in  the  commander's  world 

Battlefield  command  is  often  associated  with  severe  stress,  much  of  which  may  be  related 
to  decision  making  under  uncertainty.  In  the  battlefield,  decisions  may  determine  matters  of 
life  and  death  and  making  such  decision  on  the  basis  of  imperfect  information  (i.e., 
uncertainty)  may  be  very  stressful.  (Leith,  2003). 

The  commanders  are  at  the  focal  point  of  operational  activities.  They  communicate  with 
their  superiors,  coordinate  with  peers  and  command  their  subordinate  forces.  Intensive 
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activities  and  tight  schedules  impose  high  workload.  Commanders  may  not  have  the  time 
and  mental  resources  for  small  details;  hence,  they  may  be  able  to  use  existing  information 
only  if  it  is  clear,  readily  available  and  properly  designed  and  displayed  (Pascual  & 
Henderson,  1997;  Flach  &  Kuperman,  2001;  ). 

Representation  of  the  information  world  in  C3I  systems 

Commanders  who  operate  in  headquarters  or  air  operations  centers  do  not  have  direct 
access  to  real-world  information.  Instead,  they  must  rely  on  information  that  is  acquired  in 
the  real  world  by  various  devices,  processed  by  humans  and  computers,  distributed  via 
communication  networks  and  presented  on  C  I  information  systems. 

A  typical  C  I  information  system  may  contain  some  or  all  of  the  following  capabilities. 

■  The  basic  display  may  consist  of  some  type  of  map  or  aerial  photo  that  contains  three- 
dimensional  terrain  infonnation. 

■  A  variety  of  maps,  of  various  scales,  and  supported  by  pan  and  zoom  capabilities  to 
enable  different  views. 

*  It  may  also  be  possible  to  analyze  the  terrain  (e.g.,  compute  lines  of  sight),  analyze 
flight  route,  assess  “trafficability,”  etc. 

■  On  top  of  the  map,  various  layers  of  information  may  be  displayed,  e.g.,  enemy  forces, 
friendly  forces,  sites,  deployments,  etc. 

■  The  system  may  also  be  used  for  mission  planning,  e.g.,  monitoring  intelligence, 
viewing  raw  data  (e.g.,  UAV  videos),  preparing  flight  plans,  preparing  mission  plans 
and  orders,  and  distributing  them  via  the  communication  network. 

■  In  addition,  the  system  may  contain  general  information  (e.g.,  the  hierarchical  structure 
of  forces,  alphanumeric  intelligence,  technical  infonnation,  etc.). 

■  The  systems  are  connected  via  data  communication  networks  that  enable  each  of  the 
network  members  to  view  relevant  information.  Friendly  forces  may  transmit  to  each 
other  their  location,  status  etc.;  some  users  may  also  feed  information  into  the  system 
(e.g.,  online  intelligence).  The  existence  of  similar  enemy  information  depends  on 
intelligence  sources. 
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Representation  and  uncertainty 

The  flow  of  information  is  presented  in  Figure  2.  Events  take  place  in  the  real  world  and 
are  captured  and  acquired  by  various  devices.  This  information  is  processed,  disseminated 
and  displayed  to  the  commanders  or  the  command  team,  who  employ  various  cognitive 
processes  (described  later  in  this  report),  make  decisions  and  take  action.  Uncertainty  exists 
all  along  the  process;  a  distinction  is  made  between  basic  uncertainty  which  stems  from 
acquisition  and  display  systems  and  uncertainty  that  is  also  affected  by  the  limitations  of 
the  human  decision  maker  (interpreted  later  in  this  report). 
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Figure  2:  The  flow  of  information  from  the  world  to  the  C  I  system  and  to  the  commanders 
who  make  decisions  and  take  actions  (see  text  for  further  interpretation)^ 

This  model  is  somewhat  similar  to  a  model  that  was  recently  presented  by  Miller  & 
Shattuck  (2004).  These  authors  distinguish  between  technological  systems  (including,  data 
in  the  environment,  data  detected  by  systems  and  data  available  in  C  system)  and  the 
perceptual  and  cognitive  processes  (including  data,  comprehension  and  projection).  Their 
model  emphasizes  the  available  information  (i.e.,  SA)  rather  than  the  missing  information 
(i.e.,  uncertainty). 

In  this  section  we  analyzed  the  real-world  information  that  a  commander  or  command  team 
needs  during  various  stages  of  a  representative  C  I  mission.  This  information  is  the  basis 
for  decision  making  that  may  take  place  under  highly  stressful  conditions.  The  human 
operator  is  unable  to  directly  acquire  real-world  information  and  has  to  rely  on  inherently 
partial  and  inaccurate  (i.e.,  uncertain)  information  that  is  symbolically  represented  by  C  l 
information  systems.  As  a  result,  commanders'  decisions  are  affected  by  various  types  and 
sources  of  uncertainty  that  were  described  as  basic  uncertainty  or  as  uncertainty. 

In  the  next  section  we  present  the  theoretical  background  for  the  cognitive  analysis  of 
decision-making  in  highly  complex  C  I  situations. 
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Theoretical  background 

This  section  focuses  on  three  integrative  theoretical  concepts:  Situation  awareness  (SA), 
Sensemaking  and  Naturalistic  decision  making  (NDM).  These  concepts  have  evolved  in 
recent  years  and  provide  tools  for  research  and  analysis  of  complex  human-machine 
systems. 

Situation  Awareness  (SA) 

SA  is  a  theoretical  concept  that  enables  the  description  and  analysis  of  whole  situations. 

The  concept  may  be  useful  if  based,  consistently,  on  a  clear  definition  (Pew,  2000). 

Endsley  (1987,  1995)  proposed  a  widely  accepted  definition  of  SA  as  "the  Perception  of  the 
elements  in  environment  within  a  volume  of  time  and  space,  the  Comprehension  of  their 
meaning  and  the  Projection  of  their  status  in  the  near  future".  This  definition  applies  well  to 
physical  environments  (space,  ground,  air)  and  is  useful  for  the  analysis  of  human  activities 
in  dynamic  physical  environments  (e.g.,  command  and  control). 

Endsley's  characterization  of  the  components  of  the  SA  definition  is  task  dependent  and  is 
based  on  examples.  The  following  is  the  proposed  interpretation  of  these  components  in  the 
context  of  C  T: 

Situations:  Continuous  activities  within  a  scenario  may  be  viewed  as  a  chain  in  which 
each  link  is  a  situation.  One  situation  lasts  as  long  as  changes  in  the  scenario  are 
moderate  and  continuous;  whereas  large,  discrete  changes  may  require  the 
classification  of  a  new  situation.  For  example,  the  transition  from  the  planning  phase  to 
the  mission  performance  phase  in  C  I  is  bound  to  create  a  new  situation;  during 
mission  performance,  various  changes  and  transitions  may  also  delineate  different 
situations  (e.g.,  fighter  aircraft  takeoff,  navigation,  air-to-ground  attack,  aerial  combat 
and  landing). 

■j 

Space:  The  C  I  environment  is  about  real  physical  space  (ground,  air,  sea)  in  which 
action  takes  place. 


14 


Pamam 


Time:  Time  affects  the  dynamics  of  the  situation  (e.g.,  rate  of  change)  and  is  crucial  for 
Projection  (How  much  time  has  passed?  How  much  is  left?  Where  will  element  X  be  in 
t  seconds?);  in  addition,  timing  determines  the  coordination  of  multiple  elements. 

Relevant  elements:  The  relevance  of  objects  is  determined  by  the  situation,  the  mission 
and  the  operator's  tasks.  Examples  of  objects  that  are  relevant  to  C  I  information 
systems  were  presented  in  table  1  above. 

Perception:  It  is  suggested  that  Perception  refer  to  the  main  physical  attributes  of 
objects  including  detection,  recognition  and  identification  of  primary  features  (e.g., 
movement). 

Comprehension:  In  the  present  context,  Comprehension  refers  to  the  understanding  the 
significance  of  objects  and  events  (e.g.,  Does  it  pose  a  threat?  What  is  the  operational 
significance  of  a  specific  deployment?). 

Projection:  Projection  of  the  status  of  elements  in  the  near  future  may  be  related  to 
movement,  motion  (within  object)  or  other  changes  (e.g.,  status,  resources,  capabilities, 
priorities,  motivation).  Projection  reflects  the  notion  that,  most  of  the  time,  the  scenario 
evolves  continuously  and  regularly  and,  consequently,  events  are  more  or  less 
predictable. 

SA  is  only  one  of  the  mechanisms  that  take  part  in  decision  making  and  perfonnance 
processes.  Endsley  (1995)  suggested  that  SA  is  primarily  affected  by  the  environmental 
situation  but  also  by  task  and  system  variables  (e.g.,  system  features,  user  interface,  stress 
and  workload);  by  knowledge  (long-term  memory,  mental  models)  and  by  goals,  intentions 
and  expectations.  SA  affects  decision  making  which,  in  turn,  affects  performance  that 
feeds-back  into  the  environmental  situation  and  so  on.  The  whole  process  consumes 
attention  resources  and  relies  heavily  on  working  memory. 

It  is  widely  accepted  that  in  order  to  be  effective,  the  knowledge  base  that  enables  SA  must 
be  well  organized.  These  organizations  are  referred  to  as  mental  models  (Rouse  &  Morris, 
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1986;  Mogford,  1997;  Wilson  &  Rutherford,  1989;  Endsley,  2000),  as  schemata  (Lipshitz 
&  Ben-Shaul,  1997)  or  as  frames  (Klein,  Philips,  Rail  &  Peluso,  in  press).  It  should  be 
noted  that  a  commander  may  have  to  employ  several  cognitive  processes  in  order  to  handle 
the  many  types  of  incoming  infonnation  and  construct  a  consistent  mental  model.  It  may  be 
necessary,  for  example,  to  perform  spatial  extrapolations  in  order  to  project  the  future 
location  of  objects  (Cooper,  1989);  perform  mental  rotations  in  order  to  recognize  objects 
from  non-prototypical  perspectives  (Shepard  &  Metzler,  1971);  "translate"  verbal 
information  about  spatial  objects  into  spatial  representations;  and  integrate  information 
from  various  sources  and  modalities  into  a  coherent  model.  The  commander's  mental  model 
of  the  battlefield  may  have  dominant  spatial  and  visual  features  that  may  rely  on  "visual 
thinking"  or  "using  vision  to  think"  (Card,  Mackinlay  &  Shneiderman,  1999). 

SA  in  C3I  information  systems 

The  analysis  of  the  information  world  of  a  commander  in  terms  of  the  SA  levels  of 
Perception,  Comprehension  and  Projection  (Table  1),  demonstrates  how  the  concept  of  SA 
can  contribute  to  the  understanding  of  the  C  I  environment.  Several  researchers  have 

•5 

investigated  C  I  in  terms  of  SA  (e.g.,  Kaempf,  Klein,  Thordson  &Wolf,  1996;  Endsley, 
2000;  Entin  &  Entin,  2001). 

SA  is  crucial  for  command  and  control  (e.g.,  Wickens,  2000;  Rodgers,  Mogford  &  Strauch, 
2000;  Matthews  &  Shattuck,  2001).  In  view  of  Endsley's  (1995)  definition  of  SA  -  events 
take  place  within  the  physical  volume  of  the  real,  three-dimensional  space  and  evolve  in 
time  -  the  commander  must  be  aware  of  several  types  of  relevant  elements  (Table  1),  and 
must  perceive,  comprehend  and  project  their  future  location  and  status. 

As  indicated  above,  the  real  world  cannot  be  viewed  directly;  therefore,  commanders'  SA 
depends  on  various  representations  of  the  world,  including  the  C  I  information  system, 
verbal  communication  and  other  sources  (e.g.,  sensor  imagery).  Later  in  this  report  we 
examine  how  well  these  representations  may  support  commanders'  SA. 
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Sensemakinq 

Endsley's  (1995)  definition  of  SA  is  limited  to  perception,  processing  and  understanding  of 
data  and  infonnation  and  does  not  deal  with  decision  making  and  performance  processes. 
Sensemaking  is  a  broader  concept  that  addresses  these  issues.  In  addition,  Sensemaking  is 
more  strongly  oriented  towards  team  and  even  organizational  processes  (even  though  team 
SA  has  also  been  addressed,  e.g.,  Salas,  Prince,  Baker  &  Shrestha,  1995;  Endsley  &  Jones, 
2001).  Moreover,  Endsley's  notion  of  SA  tends  to  view  the  process  of  building  and 
maintaining  SA  as  a  somewhat  passive  response  to  the  situation,  whereas  Sensemaking  is 
explicitly  viewed  as  a  conscious  and  active  process,  (e.g.,  Louise  Comfort,  in  Leedom, 
2002;  Klein,  Phillips,  Rail  &  Peluso,  in  press). 

The  concept  of  Sensemaking  was  discussed  in  two  recent  meetings  of  invited  experts 
(Leedom,  2001,  2002).  A  wide  variety  of  subjects  was  presented,  including:  development 
and  maintenance  of  SA  by  teams  of  system  operators,  pilots  and  other  military  crews; 
Sensemaking  structure  and  processes  in  organizations;  factors  that  enhance  or  suppress 
Sensemaking  in  organizations  during  crisis,  transition,  and  risk  management;  the 
knowledge  structure  and  the  tools  needed  for  the  solution  of  complex  problems  by 
interdisciplinary  teams  of  experts. 

Klein  et  al.,  (in  press)  proposed  a  comprehensive  theory  of  Sensemaking.  The  authors  view 
Sensemaking  as  a  set  of  processes  that  is  initiated  when  a  person  or  an  organization 
recognizes  the  inadequacy  of  the  current  understanding  of  events.  Sensemaking  may  be  a 
response  to  surprise  or  failure  of  expectations.  Klein  et  al,  (in  press)  refer  to  'mental  model’ 
as  long  term  scripts  and  scenarios  that  have  been  learned  and  as  causal  mechanisms  that 
explain  how  things  work.  The  term  frame  is  used  to  describe  transient  or  ongoing 
processes.  The  frame  is  somewhat  similar  to  situation  assessment  (Endsley,  1995)  in  that  it 
is  a  hypothetical  construct  for  examining  and  adapting  the  understanding  of  the  world  and 
the  state  of  events.  Sensemaking  is  an  active  process  of  fitting  data  into  a  frame  and  fitting 
a  frame  around  the  data.  Neither  the  data  nor  the  frame  come  first,  the  data  evoke  and  help 
to  construct  the  frame  and  the  frame  defines,  connects  and  filters  the  data.  When  the  data  or 
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the  frame  seem  inadequate,  data  can  be  fitted  to  the  frame  or  the  frame  itself  may  be 
questioned,  elaborated,  compared  to  alternative  frames,  changed  or  replaced. 

Klein  et  al.,  (in  press)  present  characteristics  of  Sensemaking  that  may  contribute  to  the 
understanding  of  C  I  information  systems,  as  presented  in  the  next  section. 

Sensemaking  in  C3I  information  systems 

The  Sensemaking  symposium  (Leedom,  2001)  was  initiated  by  the  Command  and  Control 
Research  Program  (CCRP)  which  identified  Sensemaking  as  an  essential  cognitive  element 
of  the  military  decision  making  process.  Three  major  needs  were  identified  - 

■  First,  the  current  and  future  need  for  broader  and  more  complex  spectrums  of 
operations. 

■  Second,  in  response  to  the  new  spectrum  of  operation,  employ  new,  more  appropriate 
operational  concepts  and  command  approaches,  e.g.,  effect-based-operations  are 
replacing  historic  models  of  attrition  warfare. 

■  Third,  paralleling  the  information  revolution  in  the  commercial  sector  by  adopting 
network  centric  warfare  concepts,  as  opposed  to  platfonn-centric  warfare.  "An 
information  superiority-enabled  concept  of  operation  that  generates  increased  combat 
power  by  networking  sensors,  decision  makers,  and  shooters  to  achieve  shared 
awareness,  increased  speed  of  command,  higher  tempo  of  operations,  greater  lethality, 
increased  survivability  and  a  degree  of  self-synchronization"  (Albert,  Garstka  &  Stein, 
1999). 

Clearly,  C  I  information  systems  constitute  a  major  link  in  the  infonnation  superiority- 
enabled  concept  (Flach  &  Kuperman,  2001). 

Presently,  however,  Sensemaking  is  a  fairly  general  concept  which  provides  few  specific 
models  and  research  tools.  Klein's  et  al.,  (in  press)  Data/Frame  theory  of  Sensemaking 
provides  some  heuristics,  useful  concepts  and  initial  analysis  tools. 
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Klein  et  al.,  (in  press)  outlined  twelve  major  characteristics  of  Sensemaking,  several  of 

which  seem  relevant  to  the  operation  of  C  I  information  systems  (presented  in  parentheses 

below). 

1 .  Sensemaking  is  a  conscious  and  deliberate  activity.  (As  are  commanders'  actions  in  the 
C  I  environment) 

2.  Sensemaking  may  be  triggered  by  surprise  or  perceived  anomaly.  (Surprise  and 
anomaly  are  part  and  parcel  of  the  battlefield  environment). 

3.  Sensemaking  is  the  process  of  fitting  data  into  a  frame  and  fitting  a  frame  around  the 
data.  (This  model  can  be  applied  to  the  C  I  environment). 

4.  Data  are  inferred  using  the  frame  rather  than  being  perceptual  primitives.  Because  data 
are  abstractions  from  the  environment  they  are,  potentially,  distortions  of  realty.  (The 
very  nature  of  the  C  I  infonnation  system  imposes  several  of  the  distortions  described 
by  Feltovich,  Spiro  &  Coulson  (1997).  In  the  complexity  of  the  battlefield  it  is 
essential  to  perceive  whole  elements  rather  than  numerous  single  events;  continuous 
processes  may  be  presented  as  discrete  steps;  dynamic  processes  may  be  presented  as 
static  ones;  simultaneous  processes  are  treated  sequentially;  complex  systems  may  be 
conceived  as  simple  casual  mechanisms;  interacting  processes  may  be  separated;  etc.). 

5.  Data  have  to  be  distinguished  from  noise.  (This  is  definitely  the  case  in  an  environment 
which  may  contain  decoys,  destroyed  targets,  etc.). 

6.  The  frame  is  also  inferred  based  on  a  few  key  data  elements  that  serve  as  anchors.  (The 
complexity  of  the  battlefield  may  require  anchors  for  orientation  and  for  the 
Comprehension  of  the  picture  as  a  whole,  e.g.,  Wickens,  Thomas  &  Young  2000). 

7.  The  inferences  used  in  Sensemaking  do  not  conform  to  logical  deduction.  (According 
to  Klein,  et  al.,  information  operations  specialists  rely  on  quasi-logical  reasoning  rather 
than  on  deductions;  in  other  word,  inferences  tend  to  be  more  intuitive  and  are  often  a 
one-step  process). 

8.  Experts  in  an  area  reason  in  the  same  way  as  novices  (this  is  a  general  argument  that,  if 
accepted  may  applies  to  C  I  as  well  as  to  other  areas). 
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9.  Experts  have  a  richer  repertoire  of  frames  and  mental  models.  (This  has  been  shown  in 
many  areas  of  expertise  including  C  I,  e.g.,  Serfaty,  MacMillan,  Entin  &  Entin,  1997; 
Lipshitz  &  Ben-Shaul,  1997;  Vicente  &  Wang,  1998) 

10.  Sensemaking  is  used  to  achieve  functional  understanding  (what  to  do)  as  well  as 
abstract  understanding.  (Clearly,  both  are  necessary  to  the  commander  who  has  to  take 
specific  decisions  whilst  maintaining  an  overview  of  the  situation  as  a  whole). 

1 1 .  Most  of  the  mental  models  used  by  experts  and  novices  are  fragmentary.  (Battlefield 
situations  are  often  new  and  different  from  previous  situations,  hence,  it  may  be  hard  to 
develop  comprehensive  mental  models  and  the  commander  may  have  to  rely  on 
various  ensembles  of  fragmentary  models  and  frames). 

12.  The  nature  of  Sensemaking  varies  depending  on  whether  we  are  trying  to  size  up  a 
situation  (i.e.,  assessment),  build  a  richer  understanding,  raise  questions  about  our 
understanding  or  explain  away  discrepancies,  or  compare  different  explanations  or 
search  for  some  explanation  that  can  help  us  sort  out  what  is  going  on.  (This  again  is  a 
general  argument  that  relates  to  various  environments  and  activities). 

Decision  making 

In  the  chain  of  activities  of  a  commander,  or  a  commanding  team,  in  the  C  I  environment, 
decision  making  is  the  crucial  link.  The  commander's  ability  is  evaluated  primarily  by 
his/her  decision  making  capabilities,  even  though,  other  abilities  are  important  as  well  (e.g., 
leadership). 

Lipshitz,  Klein,  Orasanu  &  Salas  (2001)  reviewed  major  trends  in  the  history  of  decision 
making  and  the  development  of  Naturalistic  Decision  Making  (NDM).  Lipshitz  et  al., 
(2001)  analyzed  the  differences  between  Classical  Decision  Making  (CDM),  Behavioral 
Decision  theory  (BDT),  Judgment  and  Decision  Making  (JDM)  Organizational  Decision 
Making  (ODM)  and  Naturalistic  Decision  Making  (NDM). 

Rasmussen  (1997)  observed  that  in  decision  research  (as  well  as  in  other  human  sciences) 
there  is  a  trend  of  moving  from  normative  models  of  rational  behavior  (e.g.,  CDM), 
through  efforts  to  model  the  observed  rational  behavior  (e.g.,  JDM),  towards  focus  on 
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representing  the  actual  observed  behavior  (NDM)  and  ultimately  which  extends  to 
modeling  behavior-generating  mechanisms  such  as  system  constraints,  opportunities  and 
criteria  (e.g.,  ODM). 

For  several  reasons,  discussed  below,  NDM  seems  the  most  appropriate  approach  to  the 
analysis  of  commanders'  decision  making  in  C  I  environments. 

NDM 

Lipshitz  et  ah,  (2001)  delineated  the  main  characteristics  of  NDM.  Its  applicability  to  the 
C  I  environment  is  outlined  below. 

Proficient  decision  makers:  NDM  views  the  decision  making  processes  of  experts  as  a 
focal  point  for  research  and  investigation  of  decision  making.  In  many  important 
respects  this  is  contrary  to  the  views  of  CDM,  CDT  and  JDM  which  consider  the  level 
of  expertise  as  nuisance  variable  (Pruitt,  Cannon-Bowers  &  Salas,  1997).  Obviously, 
military  commanders,  especially  at  intennediate  levels  of  command,  are  formally 
trained  to  be  proficient  decision  makers  in  the  C  I  environment. 

Field  setting:  Zsambock  (1997)  suggested  that  "NDM  is  the  way  people  use  their 
experience  to  make  decisions  in  a  field  setting".  The  battlefield  command  post  is  a 
characteristic  example  of  such  a  setting. 

Process  orientation:  NDM  models  do  not  attempt  to  predict  which  option  will  be 
implemented,  but  describe  the  cognitive  processes  of  proficient  decision  makers. 

Situation-action  matching  decision  rules:  The  study  of  proficient  decision  makers  leads 
to  modeling  decision  making  as  matching  of  a  decision  to  a  situation,  rather  than  as 
choices  between  alternative  decisions  to  the  same  situation.  The  time  pressure  and 
stress  that  characterize  the  C  l  environment  suggest  that  at  least  during  critical  phases 
of  the  mission,  examination  of  alternatives  and  the  selection  of  the  "best"  one  may  be 
beyond  the  capacity  of  the  decision  maker  (Simon,  1978).  Several  researches  have 
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identified  situation-action  processes  in  C  I  settings  (Cohen,  Freeman  &  Wolf,  1996; 
Pascual  &  Henderson,  1997;  Drillings  &  Serfaty,  1997;  Serfaty  et  al,  1997) 

Context  bound  informal  modeling:  Proficient  decision  makers  may  be  limited  in  their 
ability  to  use  abstract  formal  models.  For  example,  Pascual  &  Henderson  (1997)  found 
that  during  C  I  decision  making,  commanders  were  unable  to  use  the  Standard 
Operating  Procedures  of  the  British  Army.  Hence,  NDM  models  depict  what 
information  decision  makers  actually  attend  to  and  which  arguments  they  actually  use, 
particularly  in  complex,  real-world,  decision  making  tasks  (Cohen  &  Freeman,  1997). 

Empirical  based  prescriptions:  JDM  and  BDT  derive  their  propositions  from  normative 
models  that  prescribe  the  "right"  decision  regardless  of  whether  or  not  such  decisions 
are  likely  to  be  taken  in  a  realistic  setting.  As  a  result,  the  ability  of  these  models  to 
predict  actual  decisions  in  complex,  real-life  settings,  is  severely  limited  (Klein,  1999). 
In  contrast,  NDM  derives  its  prescription  from  descriptive  models  of  expert 
performance.  The  applied  goal  of  such  models  may,  for  example,  be  to  improve  the 
decision  making  processes  of  novice  (less  experienced)  performers  until  they  match 
experts'  (more  experienced)  decision  making.  Many  applied  environments  draw  on  the 

•j 

experience  of  experts  rather  than  on  soundly  based  theories.  C  I  is  certainly  not  an 

•5 

exception  (Drillings  &  Serfaty,  1997),  e.g.,  much  of  the  curriculum  of  C  I  schools  is 
based  on  the  study  of  past  wars  and  battles,  on  the  personal  experience  of  veteran 
commanders  and  on  the  analysis  and  practice  of  real  and  simulated  battlefield 
scenarios  (e.g.,  Lipshitz  &  Strauss,  1997). 

■5 

Pascual  &  Henderson  (1997)  performed  a  C  I  simulation  study  with  32  subjects  who  had 
various  levels  of  command  experience.  They  analyzed  the  data  in  order  to  find  which 
decision  making  model  provided  the  best  interpretation  of  their  findings.  The  naturalistic 
strategies  accounted  for  87%  of  the  variability,  as  compared  to  classical  (2%),  hybrid  (3%) 
and  “other  models”  (8%).  The  RPD  model  (Recognition-primed  decision-making  -  see 
below)  accounted  for  60%  of  the  variability,  whilst  the  remaining  40%  were  divided 
between  eight  other  models. 


22 


Pamam 


The  RPD  Model 

The  RPD  (Recognition-primed  decision  -  Klein,  1988;  Klein,  Calderwood  &  MacGregor, 

1989;  Klein,  1997),  attempts  to  describe  what  people  actually  do  under  conditions  of  time 

pressure,  ambiguous  information,  ill-defined  goals  and  changing  conditions. 

The  1997  version  of  the  model  (Klein,  1997)  distinguishes  three  levels  of  situations;  its 

3 

relevance  to  the  C  I  environment  is  also  indicated  below. 

■  In  the  Simple  match  level,  the  decision  maker  experiences  the  situation  in  a  changing 
context  and  perceives  it  as  typical  (prototype  or  analog).  Recognition  has  four  products: 
expectancies,  relevant  cues,  plausible  goals  and  typical  actions  -  that  lead  to  the 
implementation  of  a  course  of  action.  Simple  matches  are  an  important  part  of  the 
routine  of  the  C  l  environment  (as  well  as  in  many  other  working  environments).  Much 
of  the  time  the  required  action  is  easily  selected  based  on  the  experts'  experience  and  on 
sets  of  simple  rules. 

■  In  the  Diagnose  the  situation  level,  the  complexity  lies  in  the  diagnostic  phase.  The 
decision  maker  experiences  the  situation  in  a  changing  context,  if  it  is  perceived  as 
typical,  the  process  proceeds  as  in  the  simple  match.  If  not,  a  diagnosis  may  be  made 
through  feature  matching  or  story  telling.  If  necessary,  the  decision  maker  may  seek  for 
more  data.  Recognition  has  the  same  four  by-products  as  in  the  simple  case,  however,  if 
an  anomaly  is  encountered  the  process  may  go  back  to  the  diagnosis  phase.  In  the  C  I 
environment,  difficulties  in  the  diagnostic  phase  are  an  inevitable  part  of  routine  work, 
because,  even  the  most  advanced  C  I  information  systems  leave  much  ground  for 
uncertainty. 

*  In  the  evaluate  course  of  action  level,  complexity  lies  in  the  selection  of  action.  Again, 
the  process  starts  with  the  experience  of  the  situation  in  a  changing  context  that  is 
perceived  as  typical.  Again,  recognition  has  the  same  four  by-products  (expectancies, 
relevant  cues,  plausible  goals  and  action)  in  this  case'  however,  more  than  one  action 
may  be  possible;  therefore,  each  of  the  alternative  courses  of  action  is  evaluated 
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through  mental  simulation  and  is  either  selected,  rejected  or  modified.  Finally,  a  course 
of  action  that  is  expected  to  work  is  selected  and  implemented.  The  evaluation  of  the 
adversary's  possible  courses  of  action  and  the  preparation  of  appropriate  reaction  to 

•3 

these  possibilities  is  part  of  the  C  I  routine,  especially  during  planning  phases. 

■  Obviously,  some  decisions  may  require  both  the  diagnosis  of  the  situation  and  the 
evaluation  of  courses  of  action. 

A  critical  assertion  of  the  RPD  model  is  that  people  can  use  experience  to  generate  a 
plausible  option  as  the  first  one  they  consider.  This  assertion,  as  well  as  other  assertions  of 
RPD  has  gained  considerable  empirical  support  in  a  variety  of  areas  of  expertise  (e.g.,  chess 
-  Klein,  Wolf,  Militello  &  Zsambok,  1995;  Fire  fighting  -  Klein,  1993;  C  T  -  Pascual  & 
Henderson,  1997) 

Coping  with  Uncertainty 

Lipshitz  &  Strauss  (1997)  defined  uncertainty,  as  “a  sense  of  doubt  that  blocks  or  delays 
action”.  Using  this  definition,  they  identified  three  principle  forms  of  uncertainty  in 
retrospective  reports  on  decision  making  under  uncertainty:  inadequate  understanding  (a 
sense  of  having  insufficiently  coherent  situation  awareness),  lack  of  information  (a  sense  of 
incomplete,  ambiguous  or  unreliable  information)  and  conflicted  alternatives  (a  sense  that 
available  alternatives  are  insufficiently  differentiated). 

102  students  in  a  decision-making  course  in  the  Israel  Defense  Forces  Command  and 
General  Staff  College,  were  asked  to  write  cases  of  decision  making  under  uncertainty 
based  on  personal  experience  (Lipshitz  &  Strauss,  1997).  Based  on  these  research  results, 
the  authors  proposed  the  RAWFS  (Reduction,  Assumption  based  reasoning,  Weighing  pros 
and  cons,  Forestalling,  Suppression)  Heuristic  Hypothesis  that  consists  of  quasi-nonnative 
processes  for  coping  with  uncertainty.  Decision  makers  begin  by  trying  to  reduce 
uncertainty  by  collecting  additional  information;  if  this  is  not  feasible,  they  use  assumptions 
to  fill  gaps  in  understanding;  they  compare  the  merits  of  competing  alternatives  if  such 
alternatives  are  available.  Proficient  decision  makers  may  retain  a  back-up  alternative  to 
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guard  against  undesirable  contingencies  or  suppression  (denial,  distortion  of  undesirable 
information)  may  be  used  as  a  last  resort.  This  model  is  compatible  with  various  naturalistic 
decision-making  models  (Klein,  1993;  Lipshitz,  Klein,  Orasanu  &  Salas,  2001). 


In  this  section  we  reviewed  three  comprehensive  cognitive  concepts:  SA,  Sensemaking  and 
NDM.  These  concepts,  which  developed  during  recent  years,  represent  the  efforts  to  gain 
better  psychological  understanding  on  how  people  operate  in  real-world  situations, 
particularly,  during  the  operation  of  complex,  human-machine  systems.  As  shown  above, 
each  of  the  concepts  may  contribute  to  the  analysis  and  Comprehension  of  the  C  I 
environment.  The  more  mature  notions  of  SA  and  NDM  have  already  yielded  considerable 
relevant  research;  whereas,  Sensemaking  which  is  a  very  new  and  highly  general  concept, 
does  not  lend  itself,  so  easily  to  generating  research  hypotheses.  In  the  next  section  we 
examine,  more  closely,  the  nature  of  uncertainty  in  C  l  systems. 
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Uncertainty  in  C3I  information  systems 

In  the  previous  sections  we  argued  that  C  I  information  systems  serve  as  representations  of 
the  real  world,  which  commander  cannot  view  directly.  Inevitably,  these  representations  are 
partial  and  their  accuracy  is  limited. 

Sorting  uncertainty 

Uncertainty  has  many  definitions,  Lipshitz  &  Strauss  (1997)  listed  14  different 
conceptualizations  of  uncertainty  and  similar  terms  (e.g.,  risk,  ambiguity,  turbulence, 
equivocality,  conflict).  Several  definitions  refer  to  the  way  decision  makers  conceptualize 
uncertainty  (e.g.,  Anderson,  Deane,  Hammond  &  McClelland,  1981;  Yates  &  Stone,  1992; 
Lipshitz  &  Strauss,  1997).  However,  one  issue  that  is  not  clearly  represented  by  most 
models  is  the  level  of  awareness  to  the  presence  of  uncertainty  in  the  situation  awareness 
display;  Does  the  operator  know  whether  the  available  information  represents  high  or  low 
levels  of  uncertainty? 

In  reality,  the  selection  of  coping  strategies  and  the  resulting  decisions  may  be  affected  by 
the  operator’s  awareness  of  uncertainty.  According  to  Lipshitz  &  Strauss'  (1997)  coping 
strategies,  if  an  operator  is  totally  unaware  of  the  level  of  uncertainty  he/she  may  act  as  if 
they  had  consciously  resorted  to  the  suppression  strategy.  If,  however,  the  operator  is  aware 
of  uncertainty  he/she  may  employ  some  more  effective  coping  strategies.  This  issue  can  be 
viewed  in  terms  of  the  knowledge  driven  approach,  which  argues  that  decision  making  in 
general  is  determined  by  operators’  knowledge  driven  strategies,  namely,  action  arguments 
that  describe  how  decision  makers  manipulate  domain  specific  parameters  in  order  to 
achieve  a  certain  goal  (Lipshitz  &  Rozenbaum,  in  preparation).  From  a  different 
perspective,  one  may  ask  how  to  reduce  the  gaps  between  actual  and  perceived  quality  of 
representations  (displays)  of  the  world  (e.g.,  through  indications  of  uncertainty  or 
infonnation  quality). 

3 

For  that  purpose  we  propose  two  different  sorting  approaches  for  uncertainty  in  C  I 
information  systems. 
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Sorting  by  source:  Uncertainty  can  originate  from  the  sources  of  information,  from  the 
information  display  or  from  the  limitations  of  the  user.  We  propose  to  refer  to  the  first  two 
categories  as  basic  uncertainty . 

Sorting  by  levels:  Sorting  uncertainty  according  to  its  relation  to  the  three  levels  of  SA, 
Perception,  Comprehension  and  Projection  (Endsley,  1995). 

Sorting  uncertainty  by  source 

Uncertainty  the  originates  from  technology  and  information  sources 

The  combined  sources  of  information  that  are  represented  in  the  C  I  information  system 
and  in  other  systems  (e.g.,  verbal  communication)  are  supposed  to  provide  the  commander 
with  an  accurate  and  comprehensive  representation  of  reality.  However,  even  the  finest  and 
most  advanced  acquisition  systems  cannot  warrant  full  and  accurate  real-world  information. 
Hence,  gaps  and  inconsistencies  between  reality  and  displayed  information  are  inevitable. 
The  enemy  intelligence  picture  may,  without  doubt,  be  a  major  source  of  basic  uncertainty 
(see  below  for  further  analysis). 

Uncertainty  due  to  hardware  and  software  limitations 

Events  take  place  in  a  large  and  highly  complex  real  world,  whereas,  the  C  1  information 
system  is  a  small  computer  display  system.  The  need  to  reduce  the  real  world  into  such  a 
limited  device  imposes  many  distortions.  Feltovich,  Spiro  &  Coulson  (1997)  described 
several  reductive  biases  that  may  emerge  when  experts  cope  with  tasks  that  require 
complex  cognitive  skills.  Some  of  these  mechanisms  may  also  characterize  the  disparity 
between  a  complex  real  world  and  its  reduced  C  1  information  system  depiction. 

■  The  world  is  minified  to  a  reduced  scale 

■  Most  of  the  time  the  three-dimensional  world  is  reduced  into  two-dimensional  views. 

■  Objects  are  displayed  symbolically  and  not  naturalistic  ally. 

■  Most  often  objects  have  to  be  grouped  into  large  units  (e.g.,  battalions  rather  than 
single  armored  vehicles). 
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■  When  a  high  level  of  detail  is  required,  it  is  necessary  to  "zoom-in"  and  enlarge  the 
image;  as  a  result,  only  a  small  portion  of  the  real  world  can  be  viewed  at  one  time  and 
the  world  is  fragmented  into  a  mosaic  of  small  ’’snapshots.” 

■  During  the  planning  phase  dynamic  processes  have  to  be  fragmented  into  static 
samples. 

■  Real-world  simultaneous  processes,  have  to  segmented  and  analyzed  sequentially  (e.g., 
because  they  take  place  in  different  locations). 

3 

Without  going  into  the  specifics  of  user  interfaces,  it  is  clear  that  inherent  limitations  of  C  I 
information  systems  may  add  significantly  to  basic  uncertainty. 

Uncertainty  experienced  by  the  commander 

Commanders  are  affected  by  various  additional  factors  that  may  degrade  their  decision¬ 
making  beyond  the  level  imposed  by  basic  uncertainty.  The  commander  may  lack 
expertise,  be  unprepared  for  the  current  task,  be  vulnerable  to  stress,  lack  teamwork  skills 
be  under  or  overly  confident,  etc. 

Sorting  uncertainty  by  levels  ofSA 

C  l  information  systems  are  sometimes  referred  to  as  'Situation  Awareness  Displays  (e.g., 
ISD  Data  AB,  2004),  indicating  that  one  of  their  major  roles  is  to  enable  commanders  to 
build  and  maintain  SA.  Hence,  one  may  ask  how  well  a  system  supports  SA. 

In  general,  C3I  information  systems  may  be  quite  effective  in  providing  and  representing 
Perception  level  data  and  infonnation,  but  less  so  in  supporting  the  Comprehension  and 
Projection  levels  of  SA  (Endsley,  1995).  The  reasons  for  this  are  twofold:  first, 
technologically-driven  acquisition  systems  acquire  much  data,  little  information  and  very 
little  knowledge  and  understanding  (Kuperman,  2001).  Secondly,  C3I  information  systems 
are  well  designed  for  depicting  basic,  Perception  level  data  but  not  so  well  adapted  for 
depicting  higher  levels  of  infonnation;  i.e.,  even  if  high-level  infonnation  exists,  it  may  not 
be  fully  available  to  the  user. 
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The  ” Perception  level "  of  uncertainty 

In  previous  sections  we  referred  to  "Perception"  as  pertaining  to  detection,  recognition  and 
identification  of  major  features  of  relevant  objects.  This  is  somewhat  similar  to  the  ’data' 
and  partially  to  the  'information'  level  as  defined  in  the  'cognitive  hierarchy'  of  the  US 
Army  (Kuperman,  2001).  Given  good  information  sources  and  effective  distribution 
systems,  data  may  be  quite  comprehensive  and  accurate.  Possible  remaining  sources  of 
uncertainty  are: 

The  picture  is  incomplete:  not  all  elements  are  represented  (e.g.,  elements  that  are 
hidden,  do  not  emit  radiation,  etc.). 

The  picture  is  inaccurate:  inaccuracy  can  stem  from  various  sources  -  for  example, 

■  Information  is  not  up-to-date,  data  was  updated 't'  minutes  ago. 

■  Various  sources  provide  conflicting  information. 

■  Data  sources  cannot  provide  accurate  location  information  (e.g.,  direction  is 
accurate  but  distance  is  not. 

■  Multiple  data  sources  point  to  one  object  or  to  several  closely  located  objects 
(possible  under/over  counting). 

*  Objects  may  be  detected  but  not  recognized  (e.g.,  targets  or  decoys;?  alive  or 
destroyed?). 

The  "Comprehension  level"  uncertainty 

Comprehension  (Endsley,  1995)  requires  that  data  are  transformed  into  information  and 
then  into  knowledge  (in  terms  of  the  US  Anny  cognitive  hierarchy  -  Kupennan,  2001).  In 
previous  sections  we  referred  to  Comprehension  as  pertaining  to  the  significance, 
intentions,  and  integrative  aspects  of  relevant  objects  (e.g.,  to  which  unit  does  it  belong). 

Comprehension  level  information  regarding  friendly  forces  may  often  be  obtained  (based 
on  self  reports  or  identification:  friend,  foe  or  neutral  capabilities,  etc.).  However,  the 
comprehension  level  enemy's  information  may  be  very  hard  to  acquire,  because  the 
adversary  will  do  his  best  to  hide  and  disguise  it.  What  is  the  adversary's  condition?  Does 
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he  have  sufficient  supplies?  Is  the  morale  of  the  troops  high?  And,  especially  challenging, 
What  are  his  intentions? 

Furthermore,  even  if  such  information  is  available  it  may  not  be  presented  to  the  user  in  a 
straightforward  and  readily  available  manner. 

The  "Projection  level"  uncertainty 

Projection  (Endsley,  1995)  is  about  the  “where”  and  “what”  of  relevant  elements  in  the  near 
future.  In  terms  of  the  US  Army  cognitive  hierarchy  (Kuperman,  2001),  it  is  related  to 
'understanding'. 

At  this  level  the  commander  may  experience  some  uncertainty  in  regard  to  friendly  forces 
(Do  they  follow  the  plan?  Have  they  reported  as  scheduled?)  and  much  uncertainty 
regarding  the  adversary  (Will  the  forces  continue  in  the  same  tracks?  Will  they  act 
according  to  their  doctrine?  Is  the  received  information  real  or  a  deception?).  Again,  even  if 
the  information  exists  it  may  be  difficult  to  represent  on  the  C  I  information  display. 

The  specification  of  possible  courses  of  action  of  the  adversary  represents  the  efforts  that 
are  routinely  made  by  commanders  to  reduce  uncertainty  by  anticipating  plausible 
alternatives  and  preparing  for  each  of  them  (i.e.,  “branch”  or  contingency  planning). 


Coping  with  uncertainty  in  C3I  information  systems 

The  question  is  what  can  be  done  in  order  to  reduce  basic  uncertainty  in  C  I  information 
systems  and,  particularly,  how  to  diminish  its  negative  effects.  Three  possible  approaches 
are  proposed: 

■  The  information  that  is  represented  in  the  C  l  information  system  can  become  more 
comprehensive,  more  up-to-date,  more  accurate,  etc. 

■  The  information  can  be  depicted  in  a  clear  and  readily  available  manner  and  adapted  to 
the  missions  and  to  the  needs  of  the  users. 
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■  Various  decision  support  systems  may  be  introduced  (e.g.,  Morrison,  Kelly,  Moore,  & 
Hutchins,  1998). 

Improving  information  quality  and  representation 

As  indicated  above,  C  I  information  systems  serve  as  partial  representations  of  real-world 
events.  A  more  comprehensive  and  more  accurate  representation  may  reduce  basic 
uncertainty.  Various  ongoing  projects  are  aimed  as  improving  the  technologies  that  acquire, 
distribute  and  display  battlefield  information  (e.g.,  The  USAF  Cyber  Warrior,  Kuperman, 
Whitaker  &  Brown,  2000;  DARPA's  Carnegie  Melon  Uuniversity  (CMU)  -  command  post 
of  the  future;  Lockheed  Martin  FIOP  system;  US  Marine  Corps  MAGIS  system;  US  Army 
C4I  programs).  The  details  of  such  projects  are  not  in  the  scope  of  the  present  research; 
instead,  we  briefly  review  how  technical  improvements  may  contribute  to  SA, 

Sensemaking  and  decision  making  of  C  I  information  system  users. 

Supporting  Situation  Awareness 

Endsley's  (1995)  definition  of  SA  refers  to  Perception,  Comprehension  and  Projection  of 
future  status  of  relevant  elements,  within  a  volume  of  time  and  space,  each  of  these 
components  should  be  represented  in  the  C  I  information  systems. 

The  representation  of  space 

Real  battlefield  events  take  place  in  space.  Basically,  C  l  information  systems  represent 
space  as  a  miniature  two-dimensional  entity.  Based  on  these  representations,  commanders 
need  to  understand  real  space,  visualize  it  and  run  imaginary  scenarios  in  it  (Serfaty  et  ah, 
1997).  Several  measures  can  be  taken  in  order  to  facilitate  the  visualization  of  the  world, 
e.g., 

■  Use  a  variety  of  maps  for  the  general  overview  and  aerial  photographs  for  improved 
visualizations  of  more  specific  details. 
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*  Provide  various  static  and  dynamic  visualizations  in  three-dimension,  based  on  three- 
dimensional  terrain  data  (DTM  -  digital  terrain  map),  with  the  addition  of  data  on 
human-made  objects  by  overlay  or  superposition. 

It  should  be  noted  that  basic  uncertainty,  regarding  space,  may  always  remain  high.  Scale 

'X 

differences  between  the  world  and  its  representations  are  fundamental  features  of  the  C  I 
information  system;  maps  are  rarely  up-to-date,  the  saliency  of  aerial  photographic  features 
may  depend  on  the  season  and  time  of  day  in  which  they  were  taken;  the  representation  of 
human-made  objects  may  be  lacking  (e.g.,  How  does  one  represent  the  stories  of  high 
buildings  in  urban  areas?);  in  addition,  despite  decades  of  research  and  development  (e.g., 
ISD  Data  AB,  2004)  the  availability  and  effectiveness  of  three-dimensional  displays  (e.g., 
stereoscopic  representations)  is  still  very  limited. 

The  representation  of  time 

Time  and  timing  may  be  one  the  commander's  major  concerns  during  planning  and  during 
command  and  control  (i.e.,  actual  execution  of  the  plan).  In  terms  of  SA  (Endsley,  1995) 
while  time  represents  an  important  component  in  both  Perception  and  Comprehension  it 
predominates  in  Projection.  The  timing  constraints  of  a  complex  operation  may  be  as 
difficult  as  the  design  of  a  railroad  timetable  and  even  more  so  (because  trains  are  much 
more  predictable  than  humans).  Nevertheless,  it  may  perhaps  be  possible  to  introduce 
planning  support  system  into  the  planning  modules  of  C  l  information  systems.  It  may  also 
be  possible  to  animate  proposed  solutions  in  order  to  improve  the  visualization  of  the 
temporal  elements  and  interactions  of  the  plan. 

However,  since  battlefield  timing  depends  on  human  capabilities  and  motives,  basic 
uncertainty,  regarding  timing,  is  bound  to  remain  high. 

Perception  of  elements 
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The  ability  to  acquire,  process,  distribute  and  display  information  about  relevant  objects  is 
constantly  improving  (e.g.,  Talcott  et.  al.,  2001).  A  distinction  should  be  made,  however, 
between  friendly  and  hostile  forces. 

Theoretically,  the  data  on  friendly  forces  can  be  quite  accurate.  Precise  localization  systems 
(i.e.,  GPS)  and  effective  communication  networks  can  provide  all  network  members  with 
accurate  and  up-to-date  information,  regarding  the  location  and  identity  of  friendly  forces 
and  additional  necessary  data  (e.g.,  supply  status,  technical  problems). 

The  accurate  picture  of  hostile  forces  (as  well  as  neutral  forces)  is,  obviously,  a  different 
matter.  Many  efforts  are  invested  in  developing  and  operating  a  variety  of  intelligence, 
surveillance,  reconnaissance  and  damage  assessment  systems  (e.g.,  US  Marines  Air  Ground 
Intelligence  System  -  MAGIS,  2003),  some  of  which  may,  potentially,  provide  improved 
information. 

The  large  diversity  of  means  and  devices  (e.g.,  COMINT,  SIGINT,  IMINT,  HUMINT) 
extends  the  ranges  of  time  (e.g.,  day  and  night),  space  (e.g.,  land,  sea,  weather  conditions, 
far  and  close  domains),  and  type  of  data  that  may  be  acquired.  Powerful  computers  enable 
the  processing  of  the  multitude  of  data  into  a  potentially  coherent  picture. 

The  combined  use  of  various  devices  may  improve  the  ability  to  detect  and  recognize 
objects.  For  example,  the  concurrent  use  of  TV,  FLIR  and  SAR  may  significantly  improve 
the  ability  to  distinguish  decoys  from  real  targets  (Brickner,  Oettinger  &  Kuperman,  2002). 

Image  processing  algorithms  may  assist  a  human  observer  (or  replace  him/her)  in  target 
detection  and  recognition  (ATR  -  Automatic  Target  Recognition  or  Assisted  Target 
Recognition;  Cohen  &  Tolcott,  1992). 

Still,  even  the  most  advanced  armed  forces  may  not  be  able  to  acquire  a  full,  accurate 
picture  of  enemy  forces: 

■  The  adversary  hides  and  camouflages  his  devices. 
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■  The  adversary  refrains  from  emitting  radiation  thereby  reducing  his  electromagnetic 
signature. 

■  The  adversary  uses  decoys  (e.g.,  the  Serbians  on  Kosovo:  Beaver,  1999). 

■  Elements  are  highly  mobile  and  need  to  be  sampled  frequently  for  accurate  location 
and  status. 

■  Elements  have  very  low  signatures  (e.g.,  small  terror  groups). 

■  Elements  operate  in  a  noisy  environment  (e.g.,  terror  groups  in  a  civilian 
environment). 

■  Conditions  (e.g.,  weather)  degrade  the  effectiveness  of  some  acquisition  devices, 
thereby  reducing  the  completeness  and  accuracy  of  the  picture  as  a  whole. 

Comprehension  and  Projection  of  elements 

As  indicated  above,  most  C  I  information  systems  provide  only  little  information  to  support 
the  Comprehension  and  Projection  levels  of  SA.  Hence,  Comprehension  and  Projection 
may  be  based  primarily  on  the  commanders'  expertise  and  their  ability  to  see  beyond  the 
data,  combine  and  integrate  information  from  various  sources,  recognize  patterns  and 
understand  their  operational  meaning.  This  capability  is  required  during  mission  planning 
as  well  as  during  command  and  control  phases. 

The  C  I  information  systems  may  support  these  efforts  in  various  ways- 

■  Compensate  for  lack  of  expertise  by  presenting  information  supplements  (e.g., 
procedures,  target  signatures).  The  usage  of  such  information  may,  however,  be  beyond 
the  real-time  processing  capacity  of  the  operators  (Pascual  &  Henderson,  1997). 

■  Provide  updated  intelligence,  for  example,  COMINT  and  HUMINT  may  provide 
information  regarding  the  adversary's  conditions,  e.g.,  Is  he  requesting  reinforcement? 


*  Frequent  intelligence  updates,  including  closed  cycle  intelligence  (e.g.,  information 
from  a  UAV)  reduce  uncertainty  and  shorten  the  necessary  range  of  Projection.  The 
commander  may  have  to  project  only  short  tenn  developments  and  may  be  able  to 


34 


Pamam 


validate  his/her  decisions  and  change  them  while  it  is  still  possible  to  influence  their 
outcomes.  The  C  I  information  systems  may  facilitate  this  process  by  rapidly  displaying 
and  distributing  the  necessary  information. 

Evidently,  full  and  completely  accurate  Comprehension  and  Projection  of  enemy  activities 
may  not  be  possible. 

Supporting  Sensemaking 

The  basic  ideas  behind  the  concept  of 'Sensemaking'  are  related  to  C  l  information  systems. 
Network  centric  warfare  links  people  with  infonnation  to  achieve  greater  decision  making 
effectiveness;  these  people  need  a  common  operation  picture,  furthermore,  they  need  a 
common  Sensemaking  framework  for  translating  the  common  picture  into  actionable 
knowledge.  The  process,  as  a  whole  involves  cognitive,  social,  organizational,  technical 
and  cultural  dimensions  (Leedom,  2002). 

It  may  be  argued  that  ineffective  handling  of  uncertainty  may  hamper  Sensemaking. 
Furthermore,  different  participants  in  the  common  operating  pictures  may  work  out 
different  notions  and  develop  different  biases  about  uncertainty,  thereby  impairing  common 
Sensemaking.  Hence,  reduction  of  basic  uncertainty  and  more  effective  handling  of 
remaining  uncertainty  may  enhance  Sensemaking. 

As  of  now,  few  analysis  and  research  tools  have  been  developed  for  Sensemaking,  although 
an  initial  attempt  at  this  is  being  made  by  Klein,  et  a l  (in  press). 

We  propose  to  examine  how  C  I  information  systems  can  be  designed  to  better  serve  the 
Sensemaking  mechanisms  as  described  by  Klein,  et  al.,  (in  press),  data/frame  theory. 

1 .  Sensemaking  is  the  process  of  fitting  data  into  a  frame  and  fitting  a  frame  around  the 
data:  the  C3I  information  system  is  supposed  to  provide  the  data  within  a  frame  of 
references  that  enhances  its  understanding.  Even  though  Klein  et  a/’s  'frame'  is  a 
cognitive  concept,  we  may  perhaps  hypothesize  that  an  appropriately  designed 
information  system  may  enhance  framing  and  re-framing  processes. 

2.  Data  are  inferred  using  the  frame  rather  than  being  perceptual  primitives:  C3I 
information  systems  stand  the  danger  of  presenting  users  with  a  collection  of 
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information  primitives,  rather  than  with  "frames".  If  this  is  the  case,  then  the  C  I 
information  systems  may  not  provide  a  readily  available  representation  of  the  real 
world.  The  reductive  distortions  of  Feltovich  et  al  (1997)  may  provide  some  insights 
as  to  how  these  risks  can  be  avoided.  Continuous  processes  should  be  presented  as 
continuous  rather  than  as  discrete  steps  (i.e.,.,  provide  a  continuous  representation  of 
a  mission  plan);  dynamic  processes  should  be  presented  as  such  and  not  as  static 
representations  (i.e..  provide  high  refresh  rates);  simultaneous  events  should  be 
available  simultaneously  and  not  sequentially  (i.e.,.,  enable  concurrent  display  of 
related  events  that  are  geographically  distant);  interacting  processes  should  be 
identified  and  pointed  out  rather  than  overlooked  (i.e.,.,  the  interaction  between 
friendly  actions  and  enemy  courses  of  action). 

3.  Data  have  to  be  distinguished  from  noise:  Basic  uncertainty,  as  defined  above,  can  be 
described  as  a  type  of  noise,  hence,  its  reduction  may  serve  the  required  purpose. 

4.  The  frame  is  also  inferred  based  on  a  few  key  data  elements  that  serve  as  anchors: 
Anchors  should  be  explicitly  presented  in  the  C  I  information  system  for  orientation, 
navigation  and  Comprehension  of  the  picture  as  a  whole. 

5.  Sensemaking  is  used  to  achieve  functional  understanding  (what  to  do)  as  well  as 
abstract  understanding:  Clearly,  both  are  necessary  to  the  commander  who  has  to  take 
specific  decisions  while  maintaining  an  overview  of  the  situation  as  a  whole.  Good 

C  T  information  systems  must  provide  the  general  frame  of  reference  for  abstract 
understanding  by  also  supporting  the  functional  understanding. 

6.  Most  of  the  mental  models  used  by  experts  and  novices  are  fragmentary.  If  this  is 

'j 

indeed  true,  it  is  encombent  on  the  designer  the  C  l  information  system  to  represent 
the  necessary  variety  of  fragments  and  provide  the  user  with  the  necessary  flexibility 
to  combine  the  fragments  according  to  changing  needs.  This  permits  a  more  general 
and  more  flexible  approach  than  the  representation  of  whole,  complex  mental  models. 

7.  The  nature  of  Sensemaking  varies  depending  on  whether  we  are  trying  to  assess  a 
situation,  build  a  richer  understanding,  raise  questions  about  our  understanding  or 
explain  away  discrepancies,  compare  different  hypotheses  regarding  what  may  be 
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taking  place,  or  search  for  some  explanation  that  can  help  us  sort  out  what  is  going 
on:  The  representation  of  uncertainty  (see  below)  may  be  important  for  each  one  of 
these  processes,  although  the  modes  of  presentation  should  be  adapted  to  changing 
needs. 

Information  about  basic  uncertainty 

•5 

As  argued  above,  basic  uncertainty  will  always  exist  in  C  I  information  systems.  In 
addition,  there  may  often  be  uncertainty  about  uncertainty.  As  a  result,  the  system  may 
mislead  the  operators  to  assume  that  the  information  is  more  complete  or  more  accurate 
than  it  really  is,  or  vice  versa  lead  him/her  to  distrust  valid  data. 

Known  and  unknown  uncertainty 

The  status  of  data/information  categories  that  are  normally  not  represented  in  C  I 
information  systems  (e.g.,  projection  of  future  enemy  status)  may  be  intuitively  obvious  to 
the  operator  who  does  do  not  expect  the  display  of  such  infonnation.  In  contrast,  there  may 
be  uncertainty  in  regard  with  the  completeness  and  accuracy  of  categories  that  are 
displayed.  This  information  may  be  related  to  time,  space  and,  predominantly,  to  the 
objects'  representations  in  supporting  the  commander’s  Perception,  Comprehension  and 
Projection  of  the  situation. 

Uncertainty  about  uncertainty  may  be  of  one  or  more  of  the  following  types  - 

Misses:  Information  that  exists  in  the  world,  but  is  not  represented  by  the  C  I 
information  systems. 

False  alarms:  Infonnation  that  does  not  exists  in  the  world  but  is  represented  by  the  C  I 
information  systems  (i.e.,  noise)  or  irrelevant  information  that  is  depicted  as  relevant 
(e.g.,  decoy  or  destroyed  targets  displayed  as  valid/current  targets). 

Sources  of  inaccuracy:  Inaccuracy  can  stem  from  various  sources  - 
■  Information  has  not  been  updated  for  some  time;  its  current  temporal  or  geographic 
accuracy  is  unknown. 
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■  Inaccurate  location:  information  was  acquired  by  sources  that  do  not  provide 
accurate  location  (e.g.,  provide  only  direction). 

■  Unknown  number  of  elements:  various  sources  of  information  provide  data  about 
objects;  it  is  unclear  whether  or  not  different  sources  indicate  the  same  or  different 
objects. 

■  Uncertainty  in  regard  with  the  number  of  elements  may  also  be  caused  by  display 
clutter  (i.e.,  inability  to  distinguish  between  possibly  overlapping  symbols  of 
different  objects). 

■  Unknown  identity:  objects  may  be  detected  but  not  identified  (e.g.,  What  type  of 
object  is  it?  Is  it  real  or  a  decoy?  Is  it  active  or  destroyed?  etc.). 

System  information  about  uncertainty 

There  are  three  possibilities  regarding  the  system's  status  of  uncertainty  about  uncertainty  - 

■  The  system  has  no  information  about  uncertainty. 

■  The  system  has  general  information  that  may  reflect  on  uncertainty.  (For  example,  the 
general  quality  of  target  acquisition  sensor  data  in  a  specified  area  is  low  due  to 
weather  conditions.) 

■  The  system  has  specific  knowledge  of  uncertainty,  for  example: 

o  The  time  of  the  last  update  is  known. 

o  The  size  of  location  error  due  to  limitations  of  the  acquisition  system  is  known, 
o  Different  acquisition  systems  provide  signals  from  the  same  area.  The  system 
knows  that  is  unable  to  decide  whether  to  display  a  symbol  for  one  target  or  for 
more  than  one  target. 

o  There  is  uncertainty  regarding  recognition  /  identification  of  the  detected  entity 
(e.g.,  information  suggests  to  a  60/40  percent  probability  that  the  object  is  of 
type  X/Y  respectively). 

Display  of  information  about  uncertainty 

Essentially,  it  is  possible  to  provide  the  operator  with  information  about  known  sources  of 
basic  uncertainty . 
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General  information  about  uncertainty  can  be  displayed  in  various  forms,  e.g.  - 

■  General  notes  indicating  permanent  or  temporary  weaknesses  or  malfunction  of 
acquisition,  processing  or  distribution  systems. 

■  Verbal  or  graphical  indications  of  areas,  types  of  object,  etc.,  for  which  information  is 
unreliable. 

Specific  information  about  uncertainty  can  be  indicated  by  various  means,  which  have  to  be 
adapted  to  the  type  of  basic  uncertainty  as  well  as  to  the  needs  and  working  processes  of 
the  users.  Some  possible  examples  are  - 

■  There  may  be  a  digital  (e.g.,  time  readout)  or  graphical  (e.g.,  fading  color)  indicating 
the  time  since  the  last  updates  of  information. 

■  Circles  may  indicate  the  range  of  uncertainty  of  object  location  (e.g.,  the  target  location 
error  probability  curve  depicting  the  50%  level  of  confidence  in  location  accuracy). 

■  Symbols  that  represent  either  different  objects  or  multiple  representations  of  the  same 
object  can  be  indicated  by  a  connecting  line. 

■  The  level  of  system  confidence  in  objects’  identity  can  be  indicated. 

■  Objects  that  are  suspected  as  decoys,  destroyed  targets,  etc.  may  be  so  indicated. 

Should  basic  uncertainty  be  indicated? 

The  pros  and  cons  of  this  question  are  discussed  in  the  next  section. 

Coping  with  basic  uncertainty 

If  the  operator  is  aware  of  basic  uncertainty,  he/she  may  try  to  cope  with  it  and  prevent  its 
negative  effects. 

Pros  of  representing  basic  uncertainty 

The  aware  operator  may  employ  one  of  the  first  four  coping  techniques  as  outlined  by 
Lipshitz  &  Strauss  RAWFS  model.  Decision  makers  begin  by  trying  to  reduce  uncertainty 
by  collecting  additional  infonnation  (Reduction);  if  this  is  not  feasible,  they  use 
assumptions  to  fill  gaps  in  understanding  (Assumption-based  reasoning);  they  compare  the 
merits  of  competing  alternatives  if  such  alternatives  are  available  (Weighing  pros  and 
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cons).  Proficient  decision  makers  may  retain  a  backup  alternative  to  guard  against 
undesirable  contingencies  (Forestalling).  For  example  - 

■  If  it  is  known  that  the  location  of  a  target  is  approximate,  the  commander  may  issue  a 
reconnaissance  mission  (i.e.,  Reduction). 

■  If  the  picture  of  a  specific  area  is  known  to  be  inaccurate,  the  commander  may  act  upon 
assumption  based  reasoning  but  also  take  special  precautions  and  retain  a  backup 
alternative  (i.e.,  Forestalling). 

■  The  higher  the  level  of  basic  uncertainty,  the  more  it  is  important  to  retain  backup 
alternatives  and  compare  their  merits  (i.e.,  weighing  pros  and  cons). 

■  Essentially,  if  the  level  of  uncertainty  is  known,  it  is  possible  to  avoid  Suppression 
(which  is  a  destructive  option). 

Cons  of  representing  basic  uncertainty 

The  display  of  additional  information,  regarding  uncertainty,  may  sometimes  be 
destructive.  Flach  &  Kuperman  (2001)  indicated  that  humans  may  be  overwhelmed  by  the 
complexity  of  the  interface  and  become  hostages  of  technology  if  it  is  not  based  on  sound 
Cognitive  System  Engineering  (CSE)  principles.  More  specifically  - 

■  More  data  increases  workload,  aggravates  clutter  and  may  interfere  with  the  main  task. 
For  example,  Pascual  &  Handerson  (1997)  observed  that  commanders  were  unable  to 
use  the  Standard  Operating  Procedures  of  the  British  Anny  during  the  command  and 
control  process  and  could  benefit  from  it  only  if  they  mastered  it  before  hand. 

■  In  the  battlefield  certainty  cannot  be  achieved.  Schmitt  &  Klein  (1996)  argued  that 
information  technology  may  not  provide  the  real  solution  to  uncertainty  and  that  the 
quest  for  certainty  may  be  paralyzing,  rather  than  constructive. 

■  Information  may  not  be  constructive;  the  operator  may  not  know  what  to  do  with  it  and 
may  be  faced  by  more  alternatives  that  he/she  can  handle  (Lipshitz  &  Srauss,  1997). 

■  Additional  information  may  not  fit  into  the  operator's  data  frame  (Klein,  et  al,  in  press) 
and  may  not  contribute  to  the  construction  of  an  alternative  frame. 

■  Given  that  levels  of  basic  uncertainty  are  addressed  by  the  system,  the  operator  may 
now  assume  that  everything  is  under  control,  whereas,  the  “picture”  may  still  be 
partial. 
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The  general  conclusion  is  that  there  is  a  delicate  balance  between  the  need  for  information 
on  the  quality  and  reliability  of  C  I  information  display  and  the  need  to  refrain  from 
overloading  commanders  and  their  staffs  with  additional,  potentially  confusing, 
information.  The  latter  conclusion  holds  not  only  for  information  about  basic  uncertainty 
but  for  all  types  of  C  I  information  (i.e.,  simple  C  l  information  systems  are  sometimes 
more  useful  than  highly  sophisticated  ones). 

Attenuate  the  consequences  of  basic  uncertainty 

After  the  reduction  of  basic  uncertainty  to  a  possible  minimum,  it  is  necessary  to  try  to 
avoid  the  negative  consequences  of  remaining  uncertainty.  Serfaty  et  al.,  (1997)  analyzed 
decision  making  processes  in  the  C  l  environment  and  specified  two  major  means  for 
attenuating  possible  consequences  of  uncertainty:  robustness  and  flexibility. 

Robustness 

An  operational  plan  is  robust  if  it  is  not  too  vulnerable  to  uncertainty.  Robustness  can  be 
achieved  in  various  ways. 

■  Simplicity:  simple,  straightforward  modes  of  action  are  more  robust  than  complex, 
intricate  modes;  fewer  steps  and  fewer  components  provide  fewer  opportunities  for 
error  and  improve  the  overall  probability  of  success. 

■  Effective  planning:  an  effective  operational  plan  can  contribute  to  robustness  - 

o  Using  effective  tools  to  plan  and  verify  correct  timing, 
o  Using  means  (e.g.,  real  or  virtual  sand  [table]  model)  for  the  creating  and 
examination  of  possible  courses  of  enemy  action  and  preparing  alternative 
responses. 

o  Enabling  the  creation  and  testing  of  "local"  possible  courses  of  action,  i.e., 
handling  mission  segments  rather  than  the  mission  as  a  whole,  thereby  focusing 
on  selected  parts  and  solving  specific  problems. 

Flexibility 

Flexibility  is  the  ability  to  adapt  plans  and  modes  of  action  to  changing  circumstances. 
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■  Continuous  or  frequent  updates:  If  a  commander  can  receive  reliable  intelligence 
updates  at  any  given  moment,  then  his/her  decisions  may  be  adapted  to  changing 
circumstances,  thereby  avoiding  the  development  of  large  errors.  In  such  case  it  may  be 
best  to  act  upon  a  fairly  general  plan  and  supplement  the  specifics  at  near  real-time, 
during  the  command  and  control  phase. 

■  Flexible  planning:  good  planning  (even  general  planning)  may  relieve  workload  and 
stress  during  the  real-time  command  a  control  phase.  The  preparation  of  possible 
courses  of  action,  including  the  specification  of  checkpoints  and  criteria  for  the 
selection  of  an  alternative  plan,  may  be  very  helpful. 

Flach  &  Kuperman  (2001)  view  'flexibility'  as  contrary  to  'rigidity'  on  a  scale  of  robustness. 
In  the  information  warfare  arena,  flexibility  characterizes  formal  systems,  in  contrast  with 
the  rigidity  of  automatons.  The  authors  suggest  that  experts'  performance  may  provide 
insights  as  to  what  attributes  of  the  system  make  a  difference.  Experts  are  characterized  by 
fast  and  flexible  decision  making  capabilities.  Research  in  various  fields  of  expertise  (e.g., 
tactical  situations  -  Kobus,  Proctor,  Rank  &  Holste,  2000)  shows  that  experts  generate 
high-quality  alternatives  very  early  (often  as  the  first  option  considered),  thereby 
maintaining  flexibility  and  robustness.  The  use  of  experts'  performance  as  a  model  is  in  line 
with  the  general  view  of  NDM  (Klein,  1997). 

Summary 

In  the  present  section  we  proposed  two  different  methods  for  sorting  types  of  uncertainty  in 
C  I.  Sorting  by  source  technological  (acquisition,  processing,  and  display  systems),  or  user 
related  sources,  where  system  related  sources  are  defined  as  basic  uncertainty,  or,  sorting 
by  level  of  SA  (Perception,  Comprehension  or  Projections).  Then  we  analyzed  various 
methods  of  coping  with  uncertainty,  focusing  primarily  on  basic  uncertainty.  Basic 
uncertainty  may  sometimes  be  reduced  by  improving  information  quality  and 
representation  techniques.  Figure  3  which  is  a  revised  version  of  figure  2,  depicts  some  of 
these  notions. 
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Only  parts  of  real-world  information  are  acquired  by  various  systems,  the  gap  between  the 
whole  and  the  part  constitutes  one  source  of  basic  uncertainty.  The  acquired  information  is 
processed  and  disseminated.  This  information  contains  primarily  sources  of  data,  some 
information,  little  knowledge  and  practically  no  understanding  (Kupennan,  2001). 
Information  is  represented  by  a  limited  C  l  infonnation  system,  the  short  comings  of  which 
represent  another  source  of  basic  uncertainty.  The  infonnation  that  is  presented  to  the 
commander  or  command  team  may  provide  strong  support  for  SA  at  the  Perception  level, 
but  much  weaker  support  for  Comprehension  and  Projection.  SA  processes.  Decisions  are 
based  on  available  information.  The  major  research  question  is  how  to  attenuate  the 
negative  effects  of  uncertainty.  More  specifically:  How  to  deal  with  information  about 
uncertainty?  Should  it  or  should  it  not  be  presented  to  the  operator?  What  methods  can  be 
used  to  attenuate  the  negative  effects  of  remaining  uncertainty. . 

In  the  next  (and  last)  section,  some  ideas  for  further  research,  focusing  on  the 
representation  of  uncertainty,  are  outlined  and  briefly  discussed. 
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Situation  in  the  real  world 
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Figure  3:  The  figure  is  a  revision  of  Figure  2.  Information  flows  from  the  world  to  the  C  I 
system  and  to  the  commanders  who  make  decisions  and  take  actions.  As  information  passes 
between  sources  it  is  reduced,  limited  and  distorted,  as  represented  on  the  connecting 
arrows  (see  text  above  for  further  interpretation).  The  width  and  darkness  of  the  arrows 
schematically  represent  the  relative  quantity  and  reliability  of  information._ 
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Further  research 

It  is  proposed  to  focus  further  research  on  the  issue  of  representation  of  basic  uncertainty 
and  coping  with  remaining  uncertainty. 

•5 

It  is  proposed  to  create  a  micro-world  and  a  simulated  C  I  information  system  that  would 
serve  as  a  research  environment  for  testing  various  potentially  relevant  conceptualizations. 
At  later  stages  it  may  be  possible  to  validate  some  of  the  more  advanced  notions  during  real 
military  exercises. 

First  we  suggest  testing  the  proposition  that  Endsley’s  (1995)  classification  of  situation 
awareness  can  serve  as  a  foundation  for  identifying  different  types  and  levels  of  uncertainty 

”5 

in  C  I  systems.  Measures  of  SA  (e.g.,  SAGAT,  Endsley,  2000;  or  more  dynamic 
approaches  -  Miller  &  Shattuck,  2004)  can  be  used  to  diagnose  the  various  types  and  levels 
of  uncertainty.  These  could  then  serve  as  a  baseline  for  the  testing  of  various  strategies  for 
coping  with  uncertainty  and  in  particular,  for  testing  various  types  and  levels  of 
representation  of  uncertainty  (i.e.,  Does  information  about  uncertainty  improve  SA?). 

Secondly,  it  is  proposed  to  test  the  descriptive  validity  of  the  data/frame  notion  (Klein,  et 
ah,  in  press);  Does  the  presence  or  absence  of  information  regarding  uncertainty  affect  the 
nature  of  frames  and  framing/reframing  processes?  Specific  tools  for  the  testing  these 
issues  have  not  been  developed  yet  and  may  have  to  be  prepared  as  part  of  the  research 
effort. 

Thirdly,  the  effects  of  the  representation  of  uncertainty  on  coping  strategies  will  be  tested; 
how  does  the  nature  of  C  I  information  affect  the  selection  of  each  of  the  RAWFS  coping 
strategies  (Lipshitz  &  Strauss,  1997),  e.g.,  Does  overestimation  of  the  quality  of 
infonnation  lead  to  the  use  of  repression  strategies?  The  selected  strategies  will  be  sorted 
based  on  the  analysis  techniques  used  by  Lipshitz  &  Strauss,  (1997). 

The  combined,  overall  product  of  this  research  should  provide  a  comprehensive  conception 
on  the  issue  of  representing  uncertainty  in  C3I  information  systems  and  insights  regarding 
coping  strategies  and  techniques.  In  addition,  the  ability  to  use  exiting  measures  of 
effectiveness  (i.e.,  SAGAT),  heuristics  (i.e.,  RAWFS)  and  theoretical  concepts  (i.e., 
data/frame)  will  be  extended  and  tested. 
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GLOSSARY 


ATR 

Automatic  /  Assisted  Target  Recognition 

BDT 

C3I 

Behavior  Decision  Theory 

Command,  control,  communications  and  intelligence 

CDM 

Classical  Decision  Making 

CMU 

Carnegie  Melon  University 

COMINT 

Communication  Intelligence 

DTM 

Digital  Terrain  Mapping 

FBCB2 

Force  XXI  Battle  Command  Brigade  and  Below 

FIOP 

Family  of  Interoperable  Operational  Pictures 

FLIR 

Forward  Looking  Infrared 

GPS 

Global  Positioning  System 

HUMINT 

Fluman  Intelligence 

IMINT 

Imagery  Intelligence 

JDM 

Judgment  Decision  Making 

MAGIS 

Marine  Air  Ground  Intelligence  System 

NDM 

Naturalistic  Decision  making 

ODM 

Organizational  Decision  Making 

RAWFS 

Reduction,  Assumption  based  reasoning,  Weighing  pros  and  cons, 
Forestalling,  Suppression 

RPD 

Recognition  Primed  Decision 

SA 

Situation  Awareness 

SAGAT 

Situation  Awareness  Global  Assessment  Technique 

SAM 

Surface  Air  Missile 

SAR 

Synthetic  Aperture  Radar 

SIGINT 

Signal  Intelligence 

UAV 

Unmanned  Air  Vehicle 
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